[History Server] Prefer active RayCluster for RayService sessions - #5126
Open
zhuangzhewei09 wants to merge 1 commit into
Open
[History Server] Prefer active RayCluster for RayService sessions#5126zhuangzhewei09 wants to merge 1 commit into
zhuangzhewei09 wants to merge 1 commit into
Conversation
Author
|
@machichima I've implemented No. 1 from #5065 by preferring the active RayCluster for live RayService sessions and falling back to the pending RayCluster when the active one is unavailable. PTAL! |
machichima
reviewed
Aug 24, 2026
zhuangzhewei09
force-pushed
the
feat/select-rayservice-cluster-5065
branch
from
August 24, 2026 17:57
4652dfa to
07be803
Compare
machichima
reviewed
Aug 25, 2026
machichima
left a comment
machichima
left a comment
Collaborator
There was a problem hiding this comment.
Few comments mainly for tests. I think the test struct can be simplified after removing those test cases
zhuangzhewei09
force-pushed
the
feat/select-rayservice-cluster-5065
branch
from
August 26, 2026 07:48
07be803 to
e18b68e
Compare
6 tasks
machichima
reviewed
Aug 31, 2026
zhuangzhewei09
force-pushed
the
feat/select-rayservice-cluster-5065
branch
from
August 31, 2026 13:32
e18b68e to
cf7ebd1
Compare
When live cluster access is enabled, resolve RayJob and RayService sessions from owner status instead of listing RayClusters by owner labels. Use the RayJob cluster name directly, and check the active RayService cluster before the pending cluster. Grant the History Server get access to RayJobs and RayServices. Add focused tests for status-based owner resolution, active-before- pending selection, and owner lookup errors. Explicitly enable live clusters in the test fixture to respect the default-off behavior. Refs ray-project#5065. Signed-off-by: zhuangzhewei09 <zhuangzhewei09@dingtalk.com>
zhuangzhewei09
force-pushed
the
feat/select-rayservice-cluster-5065
branch
from
September 3, 2026 09:56
cf7ebd1 to
13bfca7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why are these changes needed?
A RayService can own both an active and a pending RayCluster during an upgrade. The History Server previously selected the first RayCluster returned by Kubernetes, whose list order is undefined, so it could resolve a live RayService session to the pending cluster.
This change resolves live RayJob and RayService sessions from the RayCluster names stored in their status instead of listing RayClusters by owner labels. RayJobs use
status.rayClusterName; RayServices preferstatus.activeServiceStatus.rayClusterNameand fall back tostatus.pendingServiceStatus.rayClusterNamewhen the active cluster name is missing or the referenced cluster is not found.It also grants the History Server minimal permission to get RayJobs and RayServices, and adds focused tests for exact lookup, active-to-pending fallback, missing candidates, error propagation, avoiding owner-based list calls, and historical-session fallback.
Related issue number
Part of #5065
Follow-up to #4918
Labels
doc-updates-requiredlabel.breaking-changelabel.Checks
cd historyserver && go test -race ./pkg/... ./cmd/... -parallel 4cd historyserver && go vet ./pkg/historyserver